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.^^ , Abstract. We study a class of second order variational inequalities with bilateral con- 

. ■ straints. Under certain conditions we show the existence of a unique viscosity solution 

r^ ' of these variational inequalities and give a stochastic representation to this solution. As 

an application, we study a stochastic game with stopping times and show the existence 

of a saddle point equilibrium. 
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T 1 

[~^ , 1. Introduction and preliminaries 

o ■ 

vQ ■ We study a class of second order nonlinear variational inequalities with bilateral con- 

(^ I straints. This type of inequalities arises in zero sum stochastic differential games of mixed 

■^ ■ type where each player uses both continuous control and stopping times. Under a non- 

^P I degeneracy assumption Bensoussan and Friedman 1 1 4 1 have studied this type of prob- 

f^ ■ lems. They proved the existence of a unique solution of these variational inequalities 

"t^ I in certain weighted Sobolev spaces. This result together with certain techniques from 

stochastic calculus is then applied to show that the unique solution of these inequalities is 
the value function of certain stochastic differential games of mixed type. In this paper we 
study the same class of variational inequalities without the non-degeneracy assumption. 
The non-degeneracy assumption is crucially used in the analysis of the problem in 1 1 4|. 
Thus the method used in LI .4] does not apply to the degenerate case. We study the prob- 
ed I lem via the theory of viscosity solutions. We transform the variational inequalities with 
bilateral constraints to Hamilton-Jacobi-Isaacs (HJI for short) equations associated with 
a stochastic differential game problem with continuous control only. Then using standard 
results from the theory of viscosity solutions, we show that the value function of this 
stochastic differential game with continuous control is the unique viscosity solution of 
the corresponding variational inequalities. Then for a special case we identify this unique 
viscosity solution as the value function of the stochastic game with stopping times. We 
now describe our problem. 

Let Ui,i— 1 , 2, be the compact metric spaces. Let 
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jidxd 



a-.R" XU1XU2 

We assume that: 

(Al) 

The functions b and a are bounded and continuous, a{x,ui,U2) is C^ in x uniformly with 
respect to mi , M2- The matrix a is symmetric and non-negative definite. Further there exists 
constant Ci > such that for all m, G f/, , / = 1 , 2, 



Let 



and 



\b{x,ui,U2)-b{y,ui,U2)\ <Ci\x-y\. 



r:W XU1XU2- 



V/;:M''^M, ( = 1,2. 

We assume that 

(A2) 

(i) r, i/Ai , 1//2 are bounded and continuous, 
(ii) There exists a constant C2 > such that for all x,>' G M'', (mi , M2) G Ui x U2, 

\r{x,ui,U2) - r{y,uuU2)\ + \Wl{x) - VlWI + 1^2^ - V2{y)\ 
<C2\x-y\. 

(iii) W2<Wi- 

Let H^ H ■ "''' ^ ^'' ^ B'^x'' - 



xM^'x: 



H^{x,p,X) — inf sup 



be defined by 

1 



ir{a{x,ui,U2)X) 



+ b(x,ui,U2) ■ p + r(x,Mi,M2) 



H {x,p,X) = sup inf 



-tr(fl(x,Mi,M2)X) 



+ b{x,ui,U2) ■ p + r(x,Mi,M2) 



Consider the following Hamilton-Jacobi-Isaacs variational inequalities with bilateral 
constraints 

Wiix) <v{x) <\j/i{x), Vx 1 

Xv{x) -H+{x,Dv{x),D^v{x)) = 0, if V/2W < v(x) < y/i (x) 

Xv{x) -H+(x,Dv{x),D^v(x)) > 0, if v(x) == v^2(-*^) 



Av(x) -//+(x,Dv(x),Z)2v(x)) < 0, if v(x) == v^i(x 



(1.1) 
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and 



W2(x)<v{x)<\if\{x), \fx 

Xv{x) -H^{x,Dv{x),D^v{x)) = 0, if Yiix) < v{x) < ij/iix) 

Xv{x) ~H-{x,Dvix),D^v{x)) > 0, if v(x) = \i/2{x) 

Xv{x) -~H-{x,Dv{x),D^v{x)) < 0, if v(x) = v^i(.^:) 



(1.2) 



By a classical solution of ( II. H . we mean a C^-function v satisfying ( II. 11 1. Similarly a 
classical solution of ( ll.2t is defined. 

The rest of our paper is structured as follows. In §2, we introduce the notion of viscos- 
ity solution and establish the existence of unique viscosity solutions of these variational 
inequalities by a probabilistic method. In §3, we apply these variational inequalities to 
treat a stochastic game with stopping times. We establish the existence of a saddle point 
equilibrium for this problem. Section 5 contains some concluding remarks. 

2. Viscosity solutions 

To motivate the definition of viscosity solutions of the variational inequalities we first 
prove the following result. 

Theorem 2.1. Assume (A2)(iii). A function v G C (M ) is a classical solution of dl.H ;/ 
and only if it is a classical solution of the equation 

mdiX{mm{Xv{x) - H^ {x,Dv{x),D^v{x));X{y(x) - V/2W)}; 

A(yW-</i(x))}=0. (2.1) 



Similarly a function v G C (R ) is a classical solution of M.2\ if and only if it is a 
classical solution of the equation 

rmn{max{Av(x) ~ H^ {x,Dv{x),D''v{x));X{v{x) — ^f\{x))}; 

X{v{x)-\if2[x))}=Q. (2.2) 

Proof. Let v be a classical solution of ( II. H . Suppose x is such that 1/2 (■^) < v{x) < xj/i (x). 
Then 

Xv{x) -H+{x,Dv{x),D^v{x)) ^0, v(x)-v/2W > 0, 
v(x) - Yi (x) < 0. 

Thus ( 12. 1> clearly holds in this case. Now if v{x) = i//2(x), then 

mm{Xv{x) - H+{x,Dv{x),D^v{x));X{v{x) - \i/2{x))} =0, 

and hence (12. 1> is satisfied. Finally assume v(x) = y/i (x), then 

min{Xv{x) - H+{x,Dv{x),D^v{x));X{v{x) - Wiix))} < 0, 

and hence 

max{min{Av(jt:) —H^{x,Dv{x),D^v{x));X{v{x) - xiriix))}; 
A(y(x)-v/iW)}=0. 
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Thus V satisfies (12. 1> . We now show the converse. It is clear from (12. 1> that v{x) < y/i (x). 
If v{x) = '/i(j^) for some .^c, then it clearly satisfies v{x) > V^2(-^) by (A2)(iii). Now let 
v(x) < \j/i (x). Then from ( 12. 1> . we have 

min{Av(x)-//+(x,Dv(x),D2v(x));A(v(x)-v/2W)}=0 

and hence v(x) — y/2{x) > 0. Thus for all x, we have 1/2 C-"^) < I'l-"^) < V'^i(^)- Now let 
v(x) < y/i (x). Then from the above equation, we have 

Av(x) -H+{x,Dv{x),D^v{x)) > 0. 

Similarly if v(x) > \(f2{x), we can show that 

Av(x) -~H+{x,Dv{x),D^v{x)) < 0. 

Thus V is a classical solution of dl.H . This concludes the proof of the first part. The second 
part of the theorem can be proved in a similar way. 

Remark 2.2. Under (A2)(iii), we can also show that a function v G C^(R'') is a classical 
solution of dl.H if and only if it is a classical solution of the equation 

min{max{Av(x)-//+(x,Dv(x),D^v(x));A(v(x) - V'^i(x))}; 
A(y(x)-v/2W)}=0. 

Similarly a function v G C^ (R'^) is a classical solution of (II. 2> if and only if it is a classical 
solution of the equation 

max{min{Av(x) — //^(x,Dv(x),D^v(x));A(v(x) - xffxix))}; 
A(v(x)-v/i(x))}=0. 

Theorem 12 . 1 1 motivates us to define viscosity solutions for dl.U and (II. 2> using ( 12. H 
and (12. 2> repsectively. 

DEFINITION 2.3. 

An upper semicontinuous function v : R'' ^ K is said to be a viscosity subsolution of 
dl.U if it is a viscosity subsolution of d2.1> . Similarly a lower semicontinuous function 
V : R^' ^ M is said to be a viscosity supersolution of dl.U if it is a viscosity super- 
solution of d2.1> . A function which is both sub- and super-solution of dl.U is called 
a viscosity solution of dl.U . Similarly, viscosity sub- and super-solutions of dl.2> are 
defined. 

We now address the question of showing the existence of unique viscosity solutions of 
dl.U and (II. 2> . This is done by showing that dl.U and dl.2> are equivalent to Hamilton- 
Jacobi-Isaacs equations corresponding to a stochastic differential game. 

Let (Bi , 0)2 be two symbols. We formulate a zero sum stochastic differential. In this 
game, U, is the set of controls for player /, where U, — C/,U {©,},/ =1,2. Let a {■,■,■) be 
the non-negative square root of a(-,-,-). Extend fe, a, r to 

b:R'' xUixU2^ R'', a -.W' xUi x U2^ R'''"^, 
r-.R'' xUixU2~>M., 
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respectively such that 

b{x,o}i,-)=0, b{x,-,o}2) = 0, a{x,coi,-) = Q, a{x,-,(j}2) 
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0, 



r{x,(0i,U2) = X\(fi{x) for all U2 G U2 and r(x, -,0)2) = ?i.\if2{x 



Let {Q.,^,P) be a complete probability space and W(f) a standard li-dimensional Brown- 
ian motion on it. Let ^ denote the set of all t7,-valued functions progressively measurable 
with respect to the process W{t). For (mi(-),M2(-)) £ M x £^2, consider the controlled 
stochastic differential equation 



dX{t) = b{X{t),ui{t),U2{t))dt + aiX(t),Uiit),U2it))dWit) 



1(0) = X 
Let the payoff function be defined by 

R{x,ui{-),U2{-)) =j 



(2.3) 



-It 



r(l(f),Mi(f),M2(f))df 



A strategy for the player 1 is a 'non-antiticipating' map a : j2^ — > M, i.e., for any M2, "2 S 
£/2 such that U2{s) — ^2(5) for all < 5 < f then we have a[M2]('S) = o:[m2](*),0 < s <t. 
Let f denote the set of all non-anticipating strategies for player 1 . Similarly strategies for 
player 2 are defined. Let the set of all non-anticipating strategies for player 2 be denoted 
by A. Then the upper and lower value functions are defined by 

y+(x)==sup inf_^(;c,Mi(-),i3[Mi](-)), 



V (x)= inf sup R{x,a[u2]{-),U2{-)). 



aer 



U2G-S^2 



Then we can closely follow the arguments in fT\ to show that under (Al) and (A2), y+ 
and V^, respectively, are unique viscosity solutions of 

Xv{x) ~ H+{x,Dv{x),D^v{x)) ^0 



and 



Xv{x) - H-{x,Dv{x),D^v{x)) ^0 



in the class of bounded continuous functions, where H^,H 
defined as follows: 



(2.4) 

(2.5) 

Mare 



H^{x,p,X) = inf sup 

"i<^Viu2eU2 - 



-Xi{a{x,ui,U2)X) 



+ b{x,ui,U2) ■ p + r{x,ui,U2) 



H {x,p,X) = sup sup 

112&U2 fi[&U[ 



-\i{a{x,ui,U2)X) 



+ b{x,ui,U2) ■ p + r{x,ui,U2) 



where a — OG*. We now establish the equivalence of ( II. 1> and ( II. 2> with (12. 4t and ( 12.5 
respectively. 
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Theorem 2.4. Assume (A2)(iii). A continuous function v.Mr ^M. is a viscosity solution 
of ( 12. 4t // and only if it is a viscosity solution of ( II. H . Similarly a continuous function 
V : R -^M. is a viscosity solution of i2.5i if and only if it is a viscosity solution of il.2\ . 

Proof The proof of this theorem is a simple consequence of the observation that 

H+{x,p,X) = {H+{x,p,X)\/ Xy2{x)) AX\l/i{x) (2.6) 

and 

H-{x,p,X) = (i/-(x,/7,X)AAv/i(x))vAv/2W. (2.7) 



As a consequence of this result we have the following existence and uniqueness result 
for the solutions of (II. 1> and (II. 2> . 

COROLLARY 2.5. 

Assume (Al) and (A2). Then V^ and V^ are unique vicosity solutions of dLU and ( IL2t 
respectively in the class of bounded continuous functions. 



Proof Since (12. 4> has a unique viscosity solution in the class of bounded continuous 
functions given by V +, we get by Theorem l2.4l that V + is the unique viscosity solution of 
dLU in the class of bounded continuous functions. Similarly V^ is the unique viscosity 
solution of ( IL2> in the class of bounded continuous functions. 



Remark 2.6. In the classical case, it is quite clear from the proof of Theorem 12. II that 
under (A2)(iii), ^^2 lii v < ^f\ if v is a classical solution of ( 12. U . In fact this remains 
true even in the case of viscosity solutions. Indeed assume (A2)(iii). Then any viscosity 
solution of ( 12. 1> satisfies 



(x) < v{x) < y/i (x), for all x £ ] 



W2 

Similar result holds for viscosity solutions of (12. 2> . Observe that 1/2 is a viscosity sub- 
solution of J2.U and y/i is a viscosity super-solution of (12. 1> . Thus the desired result 
follows from a general comparison principle on viscosity solutions ||2J. 



3. Application to stocliastic games 

In this section we consider a stochastic game with stopping times. We show the existence 
of a value and a saddle point equilibrium for this problem. We now describe the stochastic 
game with stopping times. 

Let b -.W' ~* W' and a ■.W' ^ W''^''. Assume that b, a are bounded and Lipschitz 
continuous. Consider the stochastic differential equation 

AX{t) = b{X{t))dt + a{X{t))m{t),t>Q^ 
X{0) ^ X /■ ^ ' 
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Here W is a d-dimensional Brownian motion on an underlying complete probability space 
{Q.,^,P). Let r,\j/i,\j/2 '■R'' -^ ^ be bounded and Lipschitz continuous functions and 
V^i > 1/2- Let A > 0. Define 



R{x,e,x)^E 



Mt 



e^^T 



{X{s))ds + s-^^'^'\Y,{X{0))xe 



r2(^(T))ZT<e] 



(3.2) 



where 0, T are the stopping times with respect to the cr-field generated by W(f ). 

Let W denote another ^-dimensional Brownian motion independent of W, constructed 
on an augmented probability space which is also denoted by {Q.,^,P) by an abuse of 
notation. Let a^ : W' -^ M''^^'' be defined by a^ — [a yld], where /^ is thed xd identity 
matrix. Now consider the following stochastic differential equation 



dxy{t) 

xy{o) 



b{xy{t)) At + ay{xy{t)) dW(f), t > 



(3.3) 



where W — \W,W]*. Player 1 tries to minimize R(x,-,-), as in (13. 2> . over stopping times 
9 (with respect to the <7-field generated by W{t)), whereas player 2 tries to maximize the 
same over stopping times T (with respect to the (7-field generated by W{t)). Note that 
these stopping times need not be finite a.s.. In other words, each player has the option of 
not stopping the game at any time. We refer to |7| for a similar treatment to stochastic 
games with stopping times. We now define the lower and upper value functions. Let 



V (x) =supinf/?(x, 0,t), 
y+(jc)== infsup7;(jc,0,T). 

The stochastic game with stopping times is said to have a value if y+ = V^. 

We now establish the existence of a value for this problem. Let H :W^ xM.^ x] 
be defined by 

H{x,p,X) = - tr {a{x)X)+b{x)-p + r{x), 
where a = <ja* . 



Theorem 3.1 . The stochastic game with stopping times has a value and a saddle point 
equilibrium. The value of this game is the unique viscosity solution in the class of bounded 
and continuous functions of the variational inequalities with bilateral constraints given 
by 



Vx 



W2{x)<w{x)<Vi{x), 

Xw{x) - H{x,Dw{x),D^w{x)) =0, ifWiix) <w{x) < y/i{x) 

Xw{x) - H{x,Dw{x),D^w{x)) > 0, ifw{x) = xiriix) 

Aw(x) -H{x,Dw{x),D^w{x)) < 0, ifw{x) = i/i(x) 



(3.4) 
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Proof. Using the results of §2, it follows that there is a unique viscosity solution w of 
i3A\ in the class of bounded continuous functions. We now identify this solution as the 
value function of the stochastic game with stopping times. 

We first assume that w is C^. Let w{x) > \l/2{x). Let 9 be any stopping times. Define 
the stopping time f by 

T = mf{t>0:xtf2{X{tj)^yv{X{t))}, 

where X(f) is a solution of O.U with the initial condition X(0) = x. Since w is a smooth 
viscosity solution of ( I3.4> . by Ito's formula, we have for any T > and stopping time 9, 



Letting T ^ °oin the above equation, we obtain 

w{x)<Ei f '''e-^V(X(i))d.9 + e-^(''^*V(X(0Af)) 

Now using the first inequality in (|^3 ^"d the definition of f in the above, it follows that 

w(jc) < R{x,9,f). 

Since 9 is arbitrary, we get 

w(x) <V^{x). 

Next let w is C^ and w{x) < Xj/i (x). Define the stopping time 9 by 

0=inf{f >0: \l/iiX{t)) = w{X{t))}, 

where X{t) is a solution of ( 13. U with the initial condition X (0) = x. Now using the fore- 
going arguments we can show that 

w{x) >V+{x). 
Thus 

w = V+ =V-. 
We now prove this result for the general case. Let Ws be the sup-convolution of w, i.e., 

w,(x)= sup(w(^)- 1^"'''' 



Then We ^w uniformly in M'' as e ^ 0, Wg are bounded, Lipschitz continuous, semicon- 
vex and satisfy a.e. on W\ 

Xw£{x) - H{x,Dw£{x),D^W£{x)) < po(x), 
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for a modulus po (see (Sj). Now let wf be the standard mollification of w^. Then wf 
are C^-^ wf — > W£ uniformly in W' andDwf (x) ^ Dwi;{x),D^Wg{x) — >D^m'e(x) for a.e. 
a: S M'^. Also wf have the same Lipschitz constant as We and for any 7 > 0, we have 

Awf (x) " ^ tr (D^wf (x)) -//(x,Dwf (x),£»2wf (x)) < po(e) 

+ ^5W + ^, (3.5) 

where ^5 are uniformly continuous. Let 

■fd 

rs{x,ui,U2) = r(x,Mi,M2)+Po(e)H ■ 

We now assume w{x) > i/2(-*^)- Define the stopping time t as before. Then wf (x) > V'2(-*^) 
for sufficiently small £ and 5. Applying Ito's formula for wf , we obtain for any T > and 
any stopping time 9, 

wf (x) < £ I ^^'^^\-^'rs{X^s)) d.T+e-^t^^^'^^'wf (X''(r A A f)) 



(3.6) 

where X^(f ) is the solution of ( 13. 3> with the initial condition X'^{0) — x. 

By a standard martingale inequality, for any 77 > 0, we can find a constant R^ such that 



P{ sup \xr{s)\>R^] <ri. 

\0<s<T J 

Let Ht) C M"^ be such that \£lr\ \ < i? and g,j ^ uniformly on B/; \ £l^ . Using this we 
can find a local modulus p\ such that 

e\ u{^\s))As < Pi (5,7). 

Jo 

Using this in ( 13. 6> . we obtain 

wf (x) < £ I r^'^^%-H(X''(i)) d,? + e-^(^^'^^^)wf (X''(r A A f))| 

+ P2(5,7) + r(^Po(e) + ^), (3.7) 

where p2 is a local modulus. Now using moment estimates |_8J, we have 
e( sup \X{s)~X\s)\^ <Cj^ 

\0<i<7' / 

for some constant C > which may depend on T and x. Now using this in ( 13. 7> . and 
passing to the limits 7^0 and then 5, £ ^ 0, we obtain 

w(x) < £ I f'''^''\-^'r{X{s)) di + e-^(^^''^*'w(X(r A A f )) 
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Now letting T ^ °o, as before, we obtain 

w(x) <V^{x). 
Similarly we can show that 

w{x) >V+{x) 

for all X such that y/i (x) > w{x). Thus the stochastic game with stopping times has a value. 
We now show that (0, f) is a saddle point equilibrium. We need to prove this only when 
y/2{x) < w{x) < y/i (x). Using the foregoing arguments, we can show that 



w{x) — E 



^%-^V(X(i))di + e-^^'^V(X(0Af)) 





-R{x,e,x). 



Clearly (0, f) constitutes a saddle point equilibrium. 



Remark li.l. The above result generalizes the optimal stopping time problem for degen- 
erate diffusions studied by Menaldi |6| . Menaldi has characterized the value function in 
the optimal stopping time problem as a maximal solution of the corresponding variational 
inequalities with one sided constraint. He has used the penalization arguments to obtain 
his results. Here we have used the method of viscosity solutions to generaUze the optimal 
stopping time problem to stochastic games with stopping times. We also wish to mention 
that Stettner 1 7 1 has studied stochastic games with stopping times for a class of Feller 
Markov processes. He has employed a penalization argument using semigroup theory. 
Thus our approach is quite different from that of Stettner. 



4. Conclusions 

We have studied a class of second order variational inequalities with bilateral constraints. 
Under certain conditions, we have showed the existence and uniqueness of viscosity solu- 
tions by transforming the variational inequalities to HJI equations corresponding to a 
stochastic differential game. Here we have confined our attention to a particular form of 
H which arises in stochastic differential games of mixed type. A general form of H can 
be reduced to this particular form by a suitable representation formula as in |5^|. Thus 
our probabiUstic method can be used to prove the existence and uniqueness of viscosity 
solutions for more general class of second order nonlinear variational inequalities with 
bilateral constraints. 

As an application, we have showed the existence of a value and a saddle point equi- 
librium for a stochastic game with stopping times. We now give a brief description of a 
stochastic differential game of mixed type where each player uses both continuous control 
and stopping times. 

Consider the following controlled stochastic differential equation 

dX(f) = b{X(t),ui{t),u2{t))At + a{X{t),ui{t),u2{t))AW{t), t>0 
X{0) = X 

(4.1) 
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Here W(f ) is a standard J-dimensional Brownian motion on an underlying complete prob- 
ability space (n,^,f ) and CJ is the non-negative square root of a. The processes u\{t) 
and U2{t) are U\ and f/2-valued control processes of players 1 and 2 respectively which 
are progressively measurable with respect to the a-field generated by W{t). Let A > be 
the discount factor. The payoff function is given by 



R{xM-),QM-).^)^E 



/ &-^'r{X{s),ui(s),U2{s))At 
Jo 



+ e-^^0'''^[wM0))Xe<r + V2{x{T))Xr<e] 



, (4.2) 



where 0, t are the stopping times with respect to the filtration generated by W(-). 

An admissible control for player 1 is a map mi(-) : [0,oo) -^ U\ which is progressively 
measurable with respect to the (7-field generated by W(-). The set of all admissible con- 
trols for player 1 is denoted by s^i . Similarly an admissible control for player 2 is defined. 
Let ^2 denote the set of all admissible controls for player 2. 

We identify two controls mi(-),mi(-) in ^/i on [0,f] if P{u\{s) — u\{s) for a.e. s G 
[0,?]) — 1. Similarly we identify the controls in ^. 

An admissible strategy for player 1 is a map a : jz^ ^ ja^i such that if U2 = M2 on [0, s] 
then a[u2] = o:[m2] for all s G [0,oo). The set of all admissible strategies for player 1 is 
denoted by F. Similarly admissible strategies for player 2 are defined. Let A denote the 
set of all admissible strategies for player 2. 

Let S^ denote the set of all stopping times. Let F denote the set of all non-anticipating 
maps a : ^2^2 — > M X -5^ for player 1 . Similarly let A denote the set of all non-anticipating 
maps j3 : M ^ ■^2/2 X ^ for player 2. Player 1 tries to minimize 7?(x, mi (•), 0,U2{-),t) over 
his admissible control mi (■) and stopping times 9, whereas player 2 tries to maximize the 
same over his admissible control M2() and stopping times T. We now define the upper and 
lower value functions of stochastic differential game of mixed type. Let 

y+(x)-sup inf /?(x,Mi(-),0,j3[Mi](-)), 
V"(jc) = inf sup R{x,a[u2]{-),U2{-),T:)- 

The functions y+ and V^ are respectively called upper and lower value functions of the 
stochastic differential game of mixed type. This differential game is said to have a value 
if both upper and lower value functions coincide. Now we make the following conjecture. 

Conjecture. The value functions y+ and V^ are unique viscosity solutions of (II. 1> and 
M.2\ respectively in the class of bounded continuous functions. 

Note that the above conjecture is true for the special case treated in §3. Analogous 
results also holds when the matrix a is independent of the control variables and is uni- 
formly elliptic I1I4I . 
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